A picture is worth a thousand words. This sentiment echoes throughout various aspects of our lives, and the world of computing is no exception. Since their inception, graphical user interfaces (GUIs) have become a standard and preferred way for millions of computer users to interact with their computers. Accordingly, increasing importance is being placed on the way in which computers visually display the user's actions and the environment in which the user may take these actions.
Typical GUIs use a number of onscreen graphical objects to visually represent functions, applications, files, menus, and a host of other features for the computing environment. The user typically uses a keyboard and/or a mouse input device to move an onscreen pointer to select a particular graphical object for action.
While this GUI has proven effective, a new step in the evolution of computing has revealed limitations in the traditional GUIs mentioned above. Specifically, the introduction of pen-based computing devices, such as the hand-held Personal Data Assistant (PDA) or the Tablet PC being introduced with an operating system produced by Microsoft Corp., has dramatically changed the way we view the GUI and the manner in which users can interact with their computers.
As one example, people very commonly use computers and GUIs to generate and edit electronic documents. These electronic documents may contain text (e.g., electronic word processors) and/or images (e.g., pictures) that are displayed on the user's screen and are available for editing and/or printing. To interact with these onscreen objects, the user typically uses a mouse input device to move an onscreen pointer to the desired object, and presses a button on the mouse and drags it to select the object (a “click-and-drag” action).
The selection of a particular object may be reflected in the GUI by a change in its appearance. For example, electronic word processing programs, such as the Microsoft WORD® word processing program, may display text as shown in FIG. 1a.
In FIG. 1a, the text “Sample Text” appears in black on a white background. The text is arranged automatically in uniform rows of text across the user's screen, where the rows of text are assigned a predefined height based on user-defined settings (e.g., the use of 10 point font, the line spacing, etc.). Upon selecting these words, for example, using a click-and-drag motion of the mouse pointer over the words, their appearance may change to that shown in FIG. 1b. In this figure, the actual selected text is now rendered in white, and the rectangular area inhabited by the text in the row is rendered in a black background that serves to distinguish the blocked selection, and to thereby visually identify the selected text to the user. The black blocked selection occupies the entire row height and serves to differentiate the selected text (top row) from the non-selected text (bottom row).
Microsoft's WORD® word processing program also allows users to apply highlight to text in a manner similar to the text selection feature shown in FIG. 1b. Using this feature, a block of text may be highlighted with a highlight color (e.g., black text on a yellow, pink, or green background, etc.) using a click-and-drag action. Different colors are used when highlighted text is selected (e.g., white text on a blue background, etc.) as compared to selection of non-highlighted text (e.g., white text on a black background, etc.). The highlighted text appears similar to the selected text shown in FIG. 1b, except the background block is in a highlighter color (not black).
Although this prior art approach to selecting and/or highlighting text works well in the uniform, line-by-line environment of traditional word processors, this approach is less effective in other environments that allow a greater degree of freedom, such as in pen-based computing devices, where text or other information is not confined to standard horizontal lines of regular size. For example, in systems where the text is handwritten (e.g., on a personal data assistant using a touch-sensitive screen or on other pen-based computing systems), a user may be permitted to write text (in electronic ink) anywhere on a page, in any orientation. The “block selection” approach discussed above with respect to FIGS. 1a and 1b may result in some confusion as to which ink strokes are actually selected and/or highlighted in these systems where input is more freeform in nature.
The prior art approach to selecting images, as opposed to the regimented text of traditional word processors, does not offer a much better solution. FIGS. 2a and 2b show examples of how an image, such as a simple diagonal line, may appear when selected. If the line is a simple vector line created, for example, using a line drawing option available in Microsoft's VISIO® drawing program, the selected line's only change in appearance is the addition of selection handles 250 at the endpoints (see FIG. 2a). If the line is an image, such as a bitmap image, its appearance upon selection changes to include the addition of a selection box 240 with handles 250 located around the periphery of the image (see FIG. 2b). There are drawbacks to each of these approaches, particularly if one seeks to extend these approaches to the freeform ink input regime of pen-based computing systems. In the approach of FIG. 2a, the mere addition of handles 250 may not clearly indicate the selected graphical object, particularly when the information on the GUI is very detailed, colorful, and/or dense. In the approach of FIG. 2b, while the selected line image may be identified adequately by the surrounding box 240 and handles 250, there is much wasted white space attributed to this selected line. This white space is wasteful, as it obscures more visual “real estate,” or displayable area of the GUI, than may be necessary to enclose a desired selection area. Additionally, selection of an excessive amount of white space may hinder the user's ability to see, modify, or add additional ink close to the selected ink stroke.
Additional difficulties may exist when using pen-based computing systems as a replacement for conventional pen and paper. One feature that users may wish to have available in such pen-based computing systems is the ability to emphasize their handwritten ink in some way, to make it stand out from the other writing or information on the electronic and/or printed page. In the world of conventional pen and paper, this task may be accomplished easily, for example, by using a highlighter pen to emphasize the desired portion of the ink to be highlighted. While this same approach also could be extended into an electronic pen-based computing system (i.e., by passing a stylus over the desired writing a second time to apply a highlight color), this procedure is somewhat cumbersome in that it requires a change of electronic pens and a second pass over the electronic ink to be highlighted. Additionally, application of a second layer of highlighter colored ink in this manner results in storage of the highlighter ink and the underlying ink it is emphasizing as two separate entities. This independent storage causes difficulties when a user seeks to move, resize, copy, paste or otherwise manipulate the highlighted electronic ink. In other words, because the underlying electronic ink and the highlighting ink are stored as two separate entities, when the user performs an operation on the electronic ink, she also must perform that same operation on the highlighting ink to be associated with it, or the highlighting feature may be lost, misapplied, incorrectly sized, or otherwise improperly applied to the page.
The above difficulties in applying electronic highlighting to electronic ink are further exacerbated when one adds the additional difficulties associated with rendering the highlighting ink and the underlying highlighted strokes when that ink (or a portion thereof) also is being selected by the user (e.g., by a click-and-drag procedure for copying, pasting, moving, resizing, or otherwise manipulating the ink). When selecting, a user must be capable of readily seeing: (a) that highlighted ink is selected and (b) the amount of highlighted ink selected. In other words, the highlighted characteristics of the underlying ink must not be obscured during an ink selection operation.
Accordingly, there is a need for an improved approach to highlighting and identifying selected ink in a GUI environment that can overcome one or more of the deficiencies identified above.